Frequentist and Bayesian Coverage Estimations for Stratified Fault-Injection*

Abstract. This paper addresses the problem of estimating the coverage of fault tolerance through statistical processing of observations collected in fault-injection experiments. In an earlier paper, we have studied various frequentist estimation methods based on simple sampling in the whole fault/activity input space and stratified sampling in a partitioned space. In this paper, Bayesian estimation methods are introduced for stratified sampling. Two methods are presented to obtain an approximation of the posterior distribution of the coverage by calculating its moments. The moments are then used to identify the type of the distribution in the Pearson distribution system, to estimate its parameters and to obtain the coverage confidence limit. Two hypothetical example systems are used to compare the validity and the conservatism of the Bayesian and frequentist estimations. It is shown that one Bayesian estimation method is valid for both examples and that its estimations are much less conservative than the frequentist ones. However, the Bayesian estimations for stratified sampling are still conservative compared to estimations for simple sampling.